Electronic apparatus, control method thereof and computer readable storage medium

ABSTRACT

An electronic apparatus, comprises a projection unit configured to project an image, a measurement unit configured to measure a distance to an object, a projection control unit configured to control, based on the distance to the object measured by the measurement unit, projection of the image by the projection unit so that the image projected onto the object has a preliminarily set actual size length.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an electronic apparatus, and a controlmethod thereof and a computer readable storage medium.

Description of the Related Art

Generally, there are known stereo cameras that use two image captureunits to capture images having parallax. Such a stereo camera capturesan image of one subject by two image capture units simultaneously, andobtains two types of images with parallax, namely a first image and asecond image. It is possible to calculate the length in the depthdirection of the subject from the two types of images captured by thestereo camera.

Japanese Patent Laid-Open No. 2001-209827 discloses a technique formeasuring the length between two points specified on a subject by a userin a three-dimensional space, using an image capture device that cancapture parallax images. Japanese Patent Laid-Open No. 2001-209827describes generating, based on three-dimensional positional informationof a subject specified by a user on a captured image displayed on adisplay panel, a scale image representing substantially the actual sizeof the subject, synthesizing the generated scale image with the capturedimage, and displaying the synthesized image.

According to Japanese Patent Laid-Open No. 2001-209827, a scale imagewhose actual size can be measured is displayed in a superimposed mannerat an arbitrary position on a display screen, whereby the user canmeasure the length of the subject. However, the scale image is displayedon the display screen, and therefore a non-photographer (e.g., a userlocated near the subject) who is not viewing the display screen cannotconfirm the measurement result. In other words, the user cannot graspthe length of an object, the positional relation (distance) withsurrounding objects while viewing the subject directly by eyesight(without viewing the display screen).

SUMMARY OF THE INVENTION

The present invention provides a technique that allows a user to view anobject directly by eyesight to grasp the length of the object.

According to one aspect of the present invention, there is provided anelectronic apparatus, comprising: a projection unit configured toproject an image; at least one memory storing a program; and one or moreprocessors which, by executing the program, function as: a measurementunit configured to measure a distance to an object; and a projectioncontrol unit configured to control, based on the distance to the objectmeasured by the measurement unit, projection of the image by theprojection unit so that the image projected onto the object has apreliminarily set actual size length.

According to another aspect of the present invention, there is provideda method of controlling an electronic apparatus comprising a projectionunit configured to project an image and a measurement unit configured tomeasure a distance to an object, the method comprising: measuring adistance to the object by the measurement unit; and controllingprojection by the projection unit so that the image projected onto theobject has a preliminarily set actual size length, based on the measureddistance to the object.

According to another aspect of the present invention, there is provideda non-transitory computer-readable storage medium storing a program thatcauses a computer of an electronic apparatus comprising a projectionunit configured to project an image and a measurement unit configured tomeasure a distance to an object to perform a control method comprising:measuring a distance to the object by the measurement unit; andcontrolling projection by the projection unit so that the imageprojected onto the object has a preliminarily set actual size length,based on the measured distance to the object.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram illustrating exemplary functionalconfiguration of an electronic apparatus according to an embodiment;

FIG. 1B is a block diagram illustrating exemplary hardware configurationof the electronic apparatus;

FIGS. 2A and 2B are block diagrams illustrating exemplary configurationsof an image capture unit according to the embodiment;

FIG. 3 is a block diagram illustrating an exemplary functionalconfiguration of an image processing unit according to the embodiment;

FIG. 4 is a flowchart illustrating an operation of the electronicapparatus according to the embodiment;

FIG. 5 illustrates an example of image capture and display state of asubject by the electronic apparatus according to the embodiment;

FIG. 6A is an explanatory diagram of distance data;

FIG. 6B illustrates an example of a scale image;

FIG. 6C is an explanatory diagram of a block integration method;

FIG. 7 illustrates a state in which a scale image is projected onto asubject;

FIG. 8 is a flowchart illustrating an operation of the electronicapparatus of a second embodiment;

FIG. 9 is a flowchart illustrating control of a projection positionaccording to the second embodiment; and

FIGS. 10A to 10D are explanatory diagrams of a block integration methodaccording to the second embodiment.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments will be described in detail with reference tothe attached drawings. Note, the following embodiments are not intendedto limit the scope of the claimed invention. Multiple features aredescribed in the embodiments, but limitation is not made an inventionthat requires all such features, and multiple such features may becombined as appropriate. Furthermore, in the attached drawings, the samereference numerals are given to the same or similar configurations, andredundant description thereof is omitted.

First Embodiment

FIG. 1A is a block diagram illustrating a major functional configurationof an electronic apparatus 100 of a first embodiment. The electronicapparatus 100 having a projection unit 108 that projects an imageincludes a distance measurement unit 11 and a projection control unit 12as a functional configuration. The distance measurement unit 11 measuresthe distance to an object 20. The projection control unit 12 controlsprojection of images by the projection unit 108 so that an image 21projected on the object 20 has a predetermined actual size length, basedon the distance to the object 20 measured by the distance measurementunit 11. In the following, exemplary configuration that implements thedistance measurement unit 11 and the projection control unit 12 will bedescribed in detail.

The first embodiment describes a configuration for measuring thedistance to and the length of an object (hereinafter, subject) whoseimage has been captured using an image capture device such as a digitalcamera, and using the result to project an image of a predeterminedactual size length onto the subject as an image of the predeterminedactual size length. In the following, although an image capture devicehaving a projection function (e.g., a form in which a projection unit ismounted to an accessory shoe of a digital camera) is described as anexample of the electronic apparatus 100, the electronic apparatus of thepresent embodiment is not limited to such an image capture device. Forexample, the electronic apparatus may be a projection device having animage capture unit, or a personal computer (PC) having an image capturedevice and a projection device connected thereto.

FIG. 1B is a block diagram illustrating exemplary hardware configurationof the electronic apparatus 100 according to the first embodiment. InFIG. 1B, an optical system 101 includes a lens group including a zoomlens and a focus lens, an aperture adjustment device, and a shutterdevice. The optical system 101 adjusts the scaling factor, focusposition, or light amount of a subject image reaching an image captureunit 102. The image capture unit 102 includes a photoelectric conversionelement such as a CCD or a CMOS sensor that photoelectrically converts alight flux of the subject having passed through the optical system 101into an electrical signal, and an A/D conversion unit configured toconvert the input image signal into a digital image.

A control unit 103, including one or more processors (CPU) and a memory,for example, controls various operations in the electronic apparatus 100by executing, by the one or more processors, predetermined programsstored in the memory. For example, the control unit 103 calculates anexposure amount for image capturing to obtain an input image having anappropriate brightness, and controls the optical system 101 (aperture,shutter speed, etc.) and the image capture unit 102 (analog gain of asensor) in order to realize the exposure amount. An image processingunit 104 performs, besides normal image processing, a process ofcalculating various feature amounts from the image of the subject. Theimage processing unit 104 can perform similar image processing not onlyon images output from the control unit 103, but also on images read froma storage unit 105. Note that a part or all of the functions of theimage processing unit 104 may be realized by the control unit 103.

The storage unit 105 has a function of storing temporary data used invarious processes, storing images, or the like. The storage unit 109 mayinclude an information recording medium using, for example, a DynamicRandom Access Memory(DRAM), a memory card having a semiconductor memorymounted therein, a package containing a rotational recording medium suchas a magneto-optical disk, or the like.

A setting unit 106 has a function of obtaining various valuesarbitrarily set by a user when performing image capture and measurement.A display unit 107 functions as an electronic view finder (EVF) bysequentially displaying images output from the image processing unit 104on a display member such as an LCD. The projection unit 108 projects animage while enlarging or reducing the image to an arbitrary size, orprojects a light beam having an arbitrary length or color.

FIG. 2A illustrates a part of an array of pixels 202 in the imagecapture unit 102. FIG. 2B illustrates an enlarged view of one of thepixels 202. In the image capture unit 102, the pixels 202 are arrangedregularly and two-dimensionally. The pixel 202 is a so-called dual pixelimage capture element having a micro lens 201 and a pair ofphotoelectric conversion units 203A and 203B.

FIG. 3 is a block diagram illustrating exemplary functionalconfiguration of the control unit 103 and the image processing unit 104.In FIG. 3, a signal processing unit 301 performs normal signalprocessing such as a noise reduction process, a development process, orthe like. In addition, the signal processing unit 301 performs a processof compressing the tonality of an image to be processed to apredetermined output range, using a tonality compression process basedon gamma conversion. An image capture information obtaining unit 302obtains various information such as image capture mode, focal length,aperture value, exposure time, which are set by the user when performingimage capture.

A feature extraction unit 303 calculates various feature amounts such asbrightness, color, edge, or the like, from the image output from thesignal processing unit 301. A distance calculation unit 304 generatesdistance data indicating a distribution of distances from the electronicapparatus 100 to the subject in the captured image, based on a phasedifference between a plurality of subject images appearing in a lightflux coming from different regions of the pupil of the optical system101. The image capture unit 102 and the distance calculation unit 304are exemplary configurations that implement the aforementioned distancemeasurement unit 11. Here, the form of distance data calculated by thedistance calculation unit 304 is not limited to the distance to thesubject, and may take the form of distribution representing image shiftamounts of a plurality of subject images obtained based on phasedifference, or may take the form of distribution of defocus amountsobtained by converting the image shift amounts into the defocus amountsbased on the K-value.

A determination unit 311 performs various determination processes usingoutputs from the feature extraction unit 303 and the distancecalculation unit 304. The projection control unit 12, which is acomponent illustrated in FIG. 1A, controls projection of images by theprojection unit 108 based on the distance calculated by the distancecalculation unit 304. In addition, the projection control unit 12controls the projection form (projection color, brightness, etc.) of animage, and whether or not to perform projection of the image by theprojection unit 108, or the like, based on the feature of the imageextracted by the feature extraction unit 303 and the distance calculatedby the distance calculation unit 304.

In the following, a projection control process of measuring the actualsize length of a subject specified by a user on the display unit 107 ofthe electronic apparatus 100, and projecting a scale image onto theactual object using the result will be described, referring to theflowchart of FIG. 4. Note that FIG. 5 illustrates the electronicapparatus 100 capturing an image of a subject 501 whose length isdesired to be measured by a user (not illustrated). In FIG. 5, an image502 depicts an image displayed on the display unit 107.

At step S401, the control unit 103 receives, from the setting unit 106,a user-specified points, the length between which is to be measured. Forexample, measurement points are specified by the user arbitrarilytouching two points on an image being displayed on the display unit 107.FIG. 5 illustrates that a user has specified a measurement point 503 anda measurement point 504 on the image 502 displayed on the display unit107 of the electronic apparatus 100.

At step S402, the distance calculation unit 304 calculates respectivedistances to the two measurement points specified at step S401. Theconfiguration of the image capture elements illustrated in FIGS. 2A and2B causes two light fluxes having passed through different pupil regionsin the optical system 101 to be incident on the photoelectric conversionunit 203A and the photoelectric conversion unit 203B. The brightnessimages obtained respectively by the photoelectric conversion unit 203Aand the photoelectric conversion unit 203B have different pupil regionsand therefore have parallax depending on the object distance position.Accordingly, a defocus amount can be calculated from the brightnessimages by known techniques such as stereo matching, or the like.

It is possible to convert a defocus amount into a distance value usingthe image formation formula of the optical system. The image distancefrom the rear principal point position of the optical system 101 to theimage capture surface is denoted by S(0) for an in-focus object andS(0)+def1 for an out-of-focus object. Here, def1 is assumed to be thedefocus amount calculated. The distance from the front principal pointposition of the optical system 101 to an in-focus object is denoted byOBJ(0) and the distance to an out-of-focus object is denoted byOBJ(def1). Accordingly, the following equation (1) holds for an in-focusobject using the focal length f of the lens of the optical system 101.

1/OBJ(0)+1/S(0)=1/f   (1)

According to equation (1), it is possible to calculate the distanceOBJ(0) to the object using S(0) and f, which are known values. Inaddition, the following equation (2) similarly holds for the distanceOBJ(def1) to an out-of-focus object.

1/OBJ(def1)+1/(S(0)+def1)=1/f   (2)

In equation (2), it is possible to calculate the OBJ(def1) using theknown value S(0) and the calculated value def1.

The image 601 in FIG. 6A illustrates an example of distance dataobtained as a result of calculating the aforementioned distance value.In the image 601, it is indicated that the brighter the region is, thecloser the distance to the electronic apparatus 100 (image capture unit102), and the darker the region is, the farther the distance from theelectronic apparatus 100 (image capture unit 102). Note that, asillustrated in FIG. 6A, the distances (subject distances) from the frontprincipal point position of the optical system 101 to two measurementpoints 503 and 504 specified by the user are denoted by za and zb,respectively. Note that the black line representing the contour part inthe image 601 is intended for clarity of illustration and does notreflect the measured distance.

Returning to FIG. 4, the determination unit 311 determines at step S403whether or not the difference between the subject distances za and zb ofthe two measurement points 503 and 504 calculated at step S402 is equalto or smaller than a predetermined value. For example, theaforementioned comparison is performed using a value that is 5% of thesmaller one of the subject distances za and zb, i.e., the subjectdistance from the closer measurement point to the electronic apparatus100, as the predetermined value. When the difference between Za and Zbis determined to be equal to or smaller than the predetermined value,the process flow proceeds to step S405, otherwise the process flowproceeds to step S404. At step S404, the control unit 103 displays, onthe display unit 107, a message prompting the user to capture an imagefrom the front of the two measurement points. Subsequently, the controlunit 103 waits until another measurement point is specified at stepS401.

As to projection of the actual-size scale image described below,projection with the correct actual size is prevented when a line segmentconnecting two specified measurement points is not orthogonal butsubstantially tilted relative to the electronic apparatus 100. Thedetermination at step S403 and the processing at step S404 describedabove are intended to avoid such an inconvenience. Note that thepredetermined value (threshold value) used at step S403 is not limitedto that described above and therefore it is conceivable to performmeasurement with a higher accuracy by using a smaller value, forexample.

When it is determined at step S403 that the difference between thesubject distances Za and Zb is equal to or smaller than thepredetermined value, the distance calculation unit 304 measures thelength between the two measurement points specified at step S401. First,the distance (number of pixels) pc between the two points on thecaptured image is calculated. Letting (xa, ya) and (xb, yb) be thecoordinates of the two points on the image, pc is calculated by thefollowing equation (3).

pc=((xb−xa){circumflex over ( )}2+(yb−ya){circumflex over( )}2){circumflex over ( )}1/2   (3)

Furthermore, the distance calculation unit 304 calculates, as thedistance to the subject, the average value z(=(Za+Zb)/2) of thedistances Za and Zb, calculated at step S402, from the front principalpoint position of the optical system 101 to the two measurement pointson the object. Letting f be the distance from the rear main pointposition of the optical system 101 to the image capture element (pixel202), and pp be the pixel pitch of the image capture element (pixel202), the length (actual distance) S between the two points can becalculated by the following equation.

S=pc×pp×z/f   (4)

Note that calculation of the length between two points on the image isnot limited to the technique using equation (4) described above. Otherinformation may be determined using any information provided that thenumber of pixels between two points and the distance information of thesubject are used.

At step S406, the projection control unit 12 calculates the scalingfactor when the projection unit 108 projects an image or a light beamonto the subject. Here, a case is described where the projection unit108 projects a scale shape that allows for measurement of the actualsize length by eyesight. FIG. 6B illustrates a scale shape to beprojected according to the present embodiment. A scale shape 621, whichdepicts an image to be actually projected by the projection unit 108, isa ruler-like shape having tick marks. In the present embodiment, thesmaller tick mark 622 of the scale shape 621 represents 20 cm in actualsize, and the larger tick mark 623 represents 100 cm in actual size.

The projection control unit 12 determines, based on the subject distanceof the measurement point calculated at step S402, the scaling factor forenlarging or reducing the projection so that the interval of tick marksturns out to be the actual size when the projection unit 108 projectsthe scale shape 621 onto the projection surface. In other words, theprojection control unit 12 adjusts the size (scaling factor) of thescale shape 621 so that the length between adjacent tick marks 622actually turns out to be 20 cm on the projection surface located at themeasured subject distance. Note that the length represented by the tickmarks is not limited to the examples described above. In addition, auser may be allowed to set the interval between the tick marks 622, theinterval between the tick marks 623, or the like, to be a suitablenumerical value for the subject to be measured.

Next, at step S407, the feature extraction unit 303 extracts the featureamount of the subject 501 from the image 502. Specifically, the featureextraction unit 303 divides the image 502 into unit blocks, andcalculates an integral value of brightness and hue at two pointsspecified at step S401 and blocks existing between the two points. FIG.6C, which is a schematic diagram illustrating the division, illustratesan image 641 resulted from dividing the image 502 into a plurality ofblocks. In addition, the blocks indicated by bold lines, among theplurality of blocks, are blocks to be subjected to calculation ofintegral values (blocks including the two measurement points specifiedat step S401 and a line segment connecting them). The feature extractionunit 303 calculates integral values of hue in target blocks, andextracts the average value of the integral values as the feature amountof the subject 501. Treating the hue obtained by the aforementionedprocess as the hue (color of the surface of the subject 501) of theregion to which the scale shape 621 is to be projected allows forappropriately determination of the color of the scale shape to beprojected.

At step S408, the projection control unit 12 determines the color of theimage of the scale shape 621 to be projected by the projection unit 108,based on the feature amount extracted by the feature extraction unit 303at step S407. For example, it is possible to make the tick markerseasier to be viewed by selecting the complementary color of the hueindicated by the average value of the hues calculated as described aboveas the color of the image to be projected. In addition, a preliminarilyset initial value such as green can be used when the calculated hue isachromatic. In addition, the projection control unit 12 may control suchthat the higher the brightness of the subject is, the more thebrightness of the image is reduced. For example, the projection controlunit 12 may perform control to raise the brightness of the image whenthe brightness of the subject is lower than a predetermined value, andreduce the brightness of the image when the brightness of the subject isequal to or larger than a predetermined value. For example, integralvalues of brightness are calculated for the target blocks describedabove and, when their average value is larger than a predeterminedvalue, the scale shape 621 is projected with a reduced brightness ofprojection by the projection unit 108. Accordingly, ease of viewing byeyesight is improved by projecting a low-brightness image when thebrightness of the subject on which an image of a scale shape isprojected is higher than a predetermined value, and projecting ahigh-brightness image when the brightness is equal to or lower than thepredetermined value.

At step S409, the projection control unit 12 determines whether or notthe distance (subject distance) to the measurement point calculated atstep S402 is equal to or larger than a predetermined distance.Specifically, the projection control unit 12 compares the larger valueof the subject distances of two points (Za and Zb), i.e., the subjectdistance of the farther measurement point, with a preliminarily setthreshold value (e.g., 10 meters). When the subject distance is larger,the process flow proceeds to step S410, otherwise the process flowproceeds to step S411.

The farther the subject to be measured is located, the deeper the depthof the scale shape 621 and the smaller the defocus amount becomes, whichresults in insufficient resolution of actual distance and reducedreliability. Accordingly, at step S410, the projection control unit 12changes the display form of the scale shape 621 in order to notify theuser thereabout. For example, the projection unit 108 changes theprojection color of the scale shape 621 to red. Accordingly, the usercan view the projection color by eyesight to recognize that the measuredvalue has a low reliability, and take measures such as adjusting thephotographing distance. Note that the manner of changing the displayform is not limited to that described above and, for example, thedisplay color may be changed to orange when the color of the subject isred, or the scale shape 621 may be projected in a flashing mannerwithout changing its color.

At step S411, the projection control unit 12 controls the projectionunit 108 to project the scale shape 621 onto the subject in accordancewith the set scaling factor and color. The projection control unit 12controls the projection unit 108 to project a scale with a lengthcorresponding to the distance between the two specified measurementpoints on a line connecting the two measurement points, in a mannerconforming along the line.

FIG. 7 illustrates a projection state of the scale shape. As illustratedin FIG. 7, the control unit 103 displays the measurement result of thelength (130 cm in the present example) between the two specifiedmeasurement points in a manner superimposed on the image 701 in thedisplay unit 107. In addition, a scale image 702 corresponding to thescale shape 621 is projected onto the subject 501 from the projectionunit 108, which the user can view by eyesight. Accordingly, a user whois not watching the display unit 107 may also confirm that the lengthbetween the two measurement points is about 130 cm.

Note that, although the length of the scale image 702 to be projected isequivalent to the actual size length between the specified measurementpoints in the embodiment described above, the length is not limitedthereto. For example, there may be a configuration that projects alength equivalent to the length of the image to be projected, which maybe arbitrarily set by the user. According to such a configuration, auser can provide a setting to project an image equivalent to one meterin a case, for example, where the user wants to confirm by eyesight arange spanning one meter on the subject.

According to the first embodiment, as has been described above, an image(scale image) of a length equivalent to the actual size length isprojected in accordance with the distance to, or the shape of thesubject. Accordingly, a person (user who cannot view the image displayedon the display unit 107) other than the photographer can also grasp thelength of the subject.

Note that, although a case has been described in the present embodimentwhere a ruler-like image (scale image 702) is projected, the shape to beprojected is not limited thereto. The projection unit 108 may be aprojector having a liquid crystal section, or may be configured toperform drawing on an object using laser such as a laser pointer or aline laser. For example, the projection unit 108 may be configured toproject two or more light spots or linear light beams. For example, theprojection unit 108 may have two light-projecting units and may beconfigured to control the two light-projecting units so as to project,onto the subject, two light beams spaced apart by an interval equivalentto a length (e.g., 1 meter) which has been arbitrarily set by the user.According to such a configuration, the projection unit 108 can be moresimply configured than that required for scaling and projecting images.In addition, linearly projecting images or light beams of an arbitrarylength allows for confirmation by eyesight how subjects are arranged,such as whether the subjects are arranged along a straight line orspaced apart by a predetermined interval.

Additionally, in the present embodiment, the projection form by theprojection unit 108 has been changed (the color has been changed in thisexample), in order to indicate that the reliability of measurement islow in a case where the distance to the subject is far. However,determination of the reliability of measurement is not limited to thatbased on the distance to the subject and, for example, the display formmay be changed by determining the reliability based on the edge amountof the subject. When calculating the distance value by the distancecalculation unit 304 at step S402, it becomes difficult to calculate thecorrelation value by stereo matching in a region where texture of thesubject is insufficient. In a case where the edge amount of the subjectis smaller than a predetermined amount, the projection control unit 12controls the projection unit 108 not to project the image, or to projectthe image with a different projection form from that in the case wherethe edge amount of the subject is equal to or larger than apredetermined amount. For example, in a case where the block integralvalue of the edge of the subject detected by the feature extraction unit303 is smaller than a predetermined amount, the projection control unit12 controls the projection unit 108 not to project the image, or toproject the image with a different color from the usual color. Note thatthe block integral value of the edge is, for example, the integral valueof the amount of the subject in the block indicated by bold lines inFIG. 6C. Such a control allows the user to grasp that the reliability ofmeasurement is low, and adjust the positions and points of measurement.

In addition, although the present embodiment has described, as theconfiguration of the distance measurement unit 11 for obtaining distanceinformation, a configuration that generates an image based on the phasedifference between a plurality of subject images generated by lightfluxes arriving from different regions of the pupil of the image captureoptical system such as that illustrated in FIGS. 2A and 2B, otherconfigurations may be used as replacements or in conjunction. Forexample, it is also conceivable to use a configuration of a stereoscopiccamera having a plurality of lenses and image capture elements as thedistance measurement unit 11 to facilitate detection of image shiftamount with a higher accuracy. In addition, the distance measurementunit 11 may be configured to measure the distance to the object 20 usinglight or sound. For example, a configuration that measures the distanceusing a Time Of Flight (TOF) camera or ultrasonic waves can improve theperformance of measuring distance to a subject with little patternvariation.

Second Embodiment

In the first embodiment, an example has been described in which thescale image is projected onto a position connecting specifiedmeasurement points. In a case where the surface of the subject betweenthe specified measurement points is not suitable for projection of theimage (e.g., an irregularity-rich case), the projected image turns outdifficult to be viewed. In a second embodiment, the position or timingof projection is controlled to facilitate the user to view the projectedimage by eyesight. FIG. 8 is a flowchart illustrating an operation ofthe electronic apparatus of the second embodiment. In the secondembodiment, the timing of projecting the image is suitably controlled atsteps S802 to S803, and the position at which the image is projected issuitably controlled at step S807.

At step S801, the control unit 103 receives two points specified by theuser on the display unit 107 as measurement points. The processing atstep S801 is similar to that at step S401. At step S802, the imagecapture information obtaining unit 302 determines whether the focus modeof the electronic apparatus 100 is in a manual focus (MF) mode in whichthe focus is manually set, or an auto focus (AF) mode in which the focusis automatically set. When it is determined to be in the MF mode, theprocess flow proceeds to step S804, or the process flow proceeds to stepS803 when it is determined to be in the AF mode.

At step S803, the projection control unit 12 determines whether or notthe AF button has been depressed by the user and the focusing operationhas been completed. When it is determined that the focusing operationhas been completed, the process flow proceeds to step S804, or theprocess flow returns to step S802 when it is determined to be before, orin the course of, focusing.

Here, the purpose of controlling the mode of focus and the focusingoperation state will be described. In the AF mode, it is conceivablethat the correct value of the distance to the subject cannot becalculated or the value is unstable. When the projection unit 108projects an image or a light beam in such a state, it becomes difficultfor the projection unit 108 to perform correct measurement or visualrecognition. Accordingly, projection for measurement is performed aftercompletion of the AF focusing operation or in the MF mode in which thefocus does not automatically change, whereas projection for measurementis not performed during focusing operation in the AF mode.

As described above, with the timing of performing distance measurementand projection being controlled, the process flow proceeds to step S804.At step S804, the distance calculation unit 304 calculates the distancefrom the image capture unit 102 to the measurement points (the twopoints specified by the user) received at step S801. As described in thefirst embodiment, the distance calculation unit 304 obtains distancedata as illustrated in FIG. 6A, and obtains the distances Za and Zb tothe measurement points.

At step S805, the distance between measurement points received at stepS801 is measured. The method for calculating the distance between twopoints is similar to that of the first embodiment (step S405). At stepS806, the projection control unit 12 calculates a scaling factor whenprojecting an image or a light beam onto a subject. The method forcalculating the scaling factor is similar to that of the firstembodiment (step S406). At step S807, the projection control unit 12performs a process of determining a position at which an image isprojected onto the subject. In the present embodiment, the projectioncontrol unit 12 obtains the degree of change of distance measured by thedistance calculation unit 304 for a plurality of regions in which animage can be projected onto the subject, and controls the projectionunit 108 to project the image onto a region of which the degree ofchange is smaller than a predetermined value. Details of the process ofdetermining such a projection position will be described referring tothe flowchart of FIG. 9.

At step S901, the distance calculation unit 304 calculates the blockintegral of distance values using the distance data (FIG. 6A) generatedat step S804. FIGS. 10A to 10D are schematic diagrams illustrating howthe block integral of distance values is calculated. The image in FIG.10A indicates a state in which the image obtained by the image captureunit 102 (including the image 601 of the distance data) is divided intoa plurality of blocks. The block group 1010 indicated by bold lines is agroup of calculation target blocks located between measurement pointsreceived at step S801. The image illustrated in FIG. 10B indicates theresult of block integral calculation of the image illustrated in FIG.10A. For each block, an integral value of distance data (integral value)is calculated.

At step S902, the distance calculation unit 304 uses the integral valuescalculated at step S901 to calculate a variance value of the integralvalues of individual blocks included in the group of calculation targetblocks located between the measurement points received at step S801. Atstep S903, the projection control unit 12 determines whether or not thevariance value calculated at step S902 is equal to or larger than apredetermined value. The purpose is to determine whether the surface ofthe subject between measurement points that turns out to be theprojection surface on which the scale image is to be projected is flatin terms of distance, or irregularity-rich. Note that a value obtainedby other methods such as difference between maximum and minimum valuesof an integral value may be used as the evaluation value, without beinglimited to variance of distance value. When it is determined at stepS903 that the variance value is not equal to or larger than apredetermined value, the process of setting the projection position isterminated. In such a case, the projection position lies between the twospecified measurement points.

When, on the other hand, it is determined at step S903 that the variancevalue is equal to or larger than a predetermined value, the process flowproceeds to step S904. At step S904, calculation of variance of distancevalues performed by the distance calculation unit 304 on the group ofcalculation target blocks (block group 1010) at steps S901 to S902 isperformed on a group of blocks adjacent to the group of calculationtarget blocks. Specifically, variance of distance values is calculatedfor a block group 1020 and a block group 1030 indicated by bold lines inFIGS. 10C and 10D. Note that the block group 1020 is a group of blocksadjacent to the group of calculation target blocks in the upwarddirection. In addition, the block group 1030 is a group of blocksadjacent to the group of calculation target blocks in the downwarddirection.

At step S905, the projection control unit 12 determines whether or notthe variance value of the group of adjacent blocks calculated at stepS904 is smaller than the variance value of the group of calculationtarget blocks calculated at step S902. When the variance value of thegroup of adjacent blocks is equal to or larger than the variance valueof the group of calculation target blocks, the process of setting theprojection position is terminated.

When, at step S905, the variance value of the group of adjacent blocksis smaller than the variance value of the group of calculation targetblocks, the process flow proceeds to step S906. At step S906, theprojection control unit 12 changes the position at which the projectionunit 108 projects the image onto the subject. The initial position atwhich the image is projected is between the two measurement pointsspecified at step S801. However, it is conceivable that projection ofthe image may make tick markers for measuring length difficult to beviewed by eyesight, when it is determined at step S903 that the varianceof defocus between the two points is equal to or larger than apredetermined value, i.e., irregularity-rich in terms of distance.Accordingly, it is possible to improve the visibility by projecting ontoan adjacent region with a smaller variance of distance values, i.e.,being flat in terms of distance. Note that, when it is determined atstep S905 that variance values of two groups of adjacent blocks are bothsmall, the position may be changed to that with the smaller variance, ormay be arbitrarily selected by the user. In a case where no improvementis expected by moving the projection position (NO at step S905), theinitial position is used as the projection position of the image by theprojection unit 108.

Returning to FIG.8, the scale image 702 is projected at step S808 ontothe subject 501, based on the scaling factor set at step S806 and theprojected position controlled at step S807. Note that, similarly to thefirst embodiment, the projection control unit 12 may control theprojection color of the projection unit 108.

As described above, according to the second embodiment, an image of theactual size length is projected while controlling the timing and theprojection position so as to facilitate the user to view, by eyesight,an image or light beam projected onto an image-captured subject.Accordingly, it is possible to stably project an image with the actualsize length onto the subject at a position in which the image is easilyvisible.

Note that, although the second embodiment controls whether or not toperform projection in accordance with the mode of focus or the operatingstate, it is also conceivable to control the projection form (color, orthe like). For example, projection of the image may be performed usingred during the focusing operation in the AF mode, or green aftercompletion of the focusing operation and during the MF mode. The user,viewing the projection form of the image, may determine whether or notit is measurable (whether or not there is problem with the measurementaccuracy).

In addition, although a configuration has been described in the secondembodiment that detects the flat part in terms of distance to determinethe projection position of the image, the image may be projected on anin-focus part. The user can grasp that there is nothing in focus when animage is not projected onto anything. It also becomes possible to movethe subject to a position onto which the image is projected (i.e., beingin focus).

In addition, the projection control unit 12 may be configured todetermine whether or not to cause the projection unit 108 to project animage, depending on whether or not the brightness of the environmentsubjected to photometry is smaller than a predetermined value, as aresult of performing photometry of the brightness of the environment.For example, the projection control unit 12 refers to the exposureamount calculated by the control unit 103 when performing image capture,and constantly causes the projection unit 108 to perform projection fora low illumination intensity environment where the brightness is lowerthan a predetermined level. With such a control, it is possible toincrease the contrast of the subject in a low intensity environment,whereby an improved focus accuracy is expected.

Although preferred embodiments of the present invention have beendescribed above, the present invention is not limited to suchembodiments, and various modifications and changes can be made withinthe scope of the gist. For example, a part or all of the imageprocessing described in the embodiments may be performed on a device(such as a computer) external to the device (such as a camera) used forimage capture.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully asanon-transitory computer-readable storage medium') to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2019-093132, filed May 16, 2019, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An electronic apparatus, comprising: a projectionunit configured to project an image; at least one memory storing aprogram; and one or more processors which, by executing the program,function as: a measurement unit configured to measure a distance to anobject; and a projection control unit configured to control, based onthe distance to the object measured by the measurement unit, projectionof the image by the projection unit so that the image projected onto theobject has a preliminarily set actual size length.
 2. The apparatusaccording to claim 1, wherein the image is a scale image having tickmarkers spaced apart by an actual size length.
 3. The apparatusaccording to claim 1, wherein the image is a line segment or apredetermined shape.
 4. The apparatus according to claim 1, wherein theone or more processors, by executing the program, further functions as asetting unit configured to cause a user to set an actual size length ofthe line segment or shape.
 5. The apparatus according to claim 1,wherein the measurement unit is configured to calculate a distance tothe object based on an optical or audio signal emitted toward theobject.
 6. The apparatus according to claim 1, wherein the one or moreprocessors, by executing the program, further functions as an imagecapture unit configured to capture an image of the object as a subject,wherein the measurement unit is configured to calculate a distance tothe subject, based on an output from the image capture unit.
 7. Theapparatus according to claim 6, wherein the projection control unit isconfigured to control projection form of an image by the projectionunit, based on at least one of brightness, color, and edge amount of thesubject.
 8. The apparatus according to claim 6, wherein the projectioncontrol unit is configured to control whether or not to performprojection of an image by the projection unit based on at least one ofbrightness, color, and edge amount of the subject.
 9. The apparatusaccording to claim 7, wherein the projection control unit is configuredto control such that the higher the brightness of the subject is, themore the brightness of the image is reduced.
 10. The apparatus accordingto claim 7, wherein the projection control unit is configured to controlto project the image with a color having a difference equal to or largerthan a predetermined degree from the color of the subject.
 11. Theapparatus according to claim 7, wherein the projection control unit isconfigured to control, in a case where an edge amount of the subject issmaller than a predetermined amount, not to project the image, or toproject the image in a different projection form from the case where theedge amount of the subject is equal to or larger than the predeterminedamount.
 12. The apparatus according to claim 1, wherein the projectioncontrol unit is configured to, in a case where the distance to theobject measured by the measurement unit is larger than a predeterminedvalue, not project the image, or project the image in a differentprojection form from the case where the distance to the object is equalto or smaller than the predetermined value.
 13. The apparatus accordingto claim 1, wherein the projection control unit is configured to obtaina degree of change of the distance measured by the measurement unit fora plurality of regions of the object onto which the image can beprojected, and control to project the image onto a region of which thedegree of change is smaller than a predetermined value.
 14. Theapparatus according to claim 1, wherein the one or more processors, byexecuting the program, further functions as an adjustment unitconfigured to automatically adjust focus of the image capture unit,wherein the projection control unit is configured to control, in a casewhere the focusing operation by the adjustment unit has not beencompleted, not to project the image, or to project the image in adifferent form from the case where the focusing operation has beencompleted.
 15. The apparatus according to claim 14, wherein theprojection control unit is configured to project the image while a modein which a user manually adjusts focus is being selected.
 16. Theapparatus according to claim 1, wherein the one or more processors, byexecuting the program, further functions as a photometry unit configuredto measure brightness of environment, wherein the projection controlunit is configured to determine whether or not to project the image inaccordance with whether or not the brightness of the environmentmeasured by the photometry unit is smaller than a predetermined value.17. A method of controlling an electronic apparatus comprising aprojection unit configured to project an image and a measurement unitconfigured to measure a distance to an object, the method comprising:measuring a distance to the object by the measurement unit; andcontrolling projection by the projection unit so that the imageprojected onto the object has a preliminarily set actual size length,based on the measured distance to the object.
 18. A non-transitorycomputer-readable storage medium storing a program that causes acomputer of an electronic apparatus comprising a projection unitconfigured to project an image and a measurement unit configured tomeasure a distance to an object to perform a control method comprising:measuring a distance to the object by the measurement unit; andcontrolling projection by the projection unit so that the imageprojected onto the object has a preliminarily set actual size length,based on the measured distance to the object.